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ABSTRACT 

The discrepancies between the comprehensive design 
for Follow Through evaluation and the program management's needs of 
the seven models in Philadelphia are discussed. The national 
evaluation has focused on pupil performance and achievement data. The 
local evaluation^ conducted since 1968, has focused on: (1) the 
assessment of the correspondence between the philosophy/strategies of 
the seven models and the representations that had been established in 
the participating schools; (2) codifying the instructional evaluation 
procedures in the classroom and the interpretation of mastery 
(criterion) scores for some of the model sponsors; find (3) the 
assessment and interpretation of an ever- increasing involvement of 
the community. Collectively, the foci of the local evaluations 
indicate that not until a program is fully implemented will there be 
requests for information describing outcomes in pupil achievement 
terms . (Author/KM) 
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ABSTRACT 



Re lationship of the Follow Through Evaluation to Local 



Needs r discusses the discrepancies between the comprehensive design 
for Follow Through evaluation and the prograun management's needs 
of the seven Models in Philadelphia. The national evaluation has 
focused on pupil performance and achievement data. The local 
evaluation effort , conducted since 1963, has in contrast atten^ted 
to undergird the obj'^ctives of the national evaluation by satisfy- 
ing the needs of model sponsors, local liaison personnel, princi- 
pals, teachers, and the project administrator. The first local 
focus was the assessment of the correspondence between the philo- 
sophy/strategies of the seven models and the representations 
that had been established in the participating schools. For this 
reason, the monitoring activities and evaluation procedures empha- 
sized concordance analysis techniques. 



response to the needs of classroom teachers who needed immediate 
feedback, in order to determine whether the methods of the model 
were producing positive effects. Therefore, the analysis proce- 
dures consisted of codifying the instructional evaluation proce- 
dures in the classroom and the interpretation of mastery (criterion) 
scores for some of the model sponsors. 

Concurrent with these conco^rns, a third focus evolved. 
The need to assess and interpret an ever increasing involvement of 



The second focus of the local evaluation developed in 
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the community necessitated the development of survey and interview 
techniques • From this community contact, evaluation guidelines 
geared to these local needs and conditions were established. These 
interactions with community resource personnel precipitated desires 
for data which covered the full spectrum of the program. 

Collectively, the foci of the local evaluations indicate 
that not until a program, even one which consists of many well- 
defined program subcomponents, is fully in^lemented will there be 
requests for information describing outcomes in pupil achievement 

terms. 
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RELA lONSHIP OF THE FOLLOW THROUGH EVALUATION 
TO LOCAL NEEDS 

Judy Goodwin 

Office of Research and Evaluation 
The School District of Philadelphia 

Project Follow Through is a federally funded program designed 
to help the nation's "culturally disadvantaged" children. It is 
an outgrowth of the recognition for the need to continue the 
gains made by pre-school children in the Head Start Program. 
Twenty different educational approaches (models) have been imple- 
mented throughout the nation, seven of which are being tried in 
Philadelphia, 

The program operates at thre.^ levels: 
1) federal, 

ii) sponsor (based at various educational institutions 

throughout the country and responsible for a partic- 
ular model) , 
iii) local. 

National evaluation efforts were designed to determine how effective 
each of the models was^ and to isolate those features which were 
the most successful. Since the inception of the prograun in 1968, 
the main focus of this evaluation has been the classical pre-post, 
eacperimental-control design and the collection of standardized 
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achievement data. As the program evolved, it became evident that 
there were discrepancies between this evaluation design, sponsor 
needs and local developments. Local evaluation efforts have, there- 
fore, attempted to bridge the gap, and in particular satisfy local 
needs. 

Patterns of or Evolution of Needs ; 
a) Year 1 ; 

As is characteristic of large-scale programs of such a 
con^lex nature, the task of implementation met with confusion and 
floundering at all levels. In mcmy cases, the philosophy and 
techniques appropriate to each model had to be learned on the job, 

since it was not known ahead of the school year which teachers were 

. . . * 

to participate in the program. Sponsor input was sporadic, no 

doubt due to the fact that they were busy building foundations too. 
The beginnings of the program found personnel resistauit to change, 
hazy about what changes they were actually expected to bring about, 
and anxious about the outcomes. 

The first local evaluation effort, therefore, atten5)ted 
to assess how closely the implementation of the seven models corre- 
sponded with the overall philosophy and objectives of the program, 
as well as some of the individual model objectives. A monitoring 
instrument was developed, which focused on classroom activities, 
instructional style, pupil response and the use of materials and 
classroom aides. 
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b) Year 2 ; 

The realization that the sponsors could not adequately meet 
the needs of the local sites because of geographic and staffing 
difficulties, resulted in the appointment of seven model liaisons. 
They became responsible for interpreting the model at the local 
level. Since the classroom component was well underway, energies 
could now be focused on implementing other con^onents of the program, 
fiuch as parental involvement and the utilization of commxinity re- 
sources to provide health and social services. 

At the federal level, evaluation activities continued to 
be the collection of pre-post achievement data, with the introduc- 
tion of a new endeavor — home interviews. At the local level, there 
was still a need to assess how well the specific model strategies 
were being represented in the classroom. This time the model 
liaisons were consulted and seven separate monitoring forms were 
developed. The monitoring activities were plamned so that immediate 
feedback could be given to the teachers. In addition, parental 
activities and the types of health services established were docu- 
mented. ^ 

c) Years 3 and 4 ; 

These years marked the continued refinement of all program 
components. The questions asked by prograun personnel were now related 
tot 

'•Goodwin, J. Follow Through Report; A Historical Development . 
The School District of Philadelphia, 1972. 



4 



i) the impact of the model strategies on pupil achievement 
and attitude, 
ii) which models were the most effective and why. 
iii) the effect of supplemental services on pupil achievement 
and attitude. 

iv) the effectiveness of the parental con^onent in bringing 
about change, 
v) institutional change. 
In short, years 3 and 4 called for data related to program effective- 
ness and impact as opposed to data related to program in5)lementation 
in the early years. 

At the national level, classroom observations supplemented 
parent interviewing and the collection of achievement data. At 
the local level, survey and interview techniques were developed to 
assess the intact of the program on all concerned. The total 
parent, teacher and aide population was surveyed and individxial 
interviews were conducted with program directors, model liaisons 
and principals. The data supplied information on the perceived 
strengths and weaknesses of each of the models and the obstacles 
yet to be overcome. 

Concurrently, due to the lack of feedback from the national 
evaluation, standardized achievement data had to be collected and 
analyzed. Although certain trends became discernible, the data 
were by no means conclusive and this effort must be continued in 
years 5 and 6. 



d) Year 5 ; 

The fact that the national evaluation has, to date, been 
unable to fulfill either sponsor or local needs, has placed in- 
creasing responsibility on each sponsor to collect the necessary 
data. They, in turn, must rely heavily on the local sites for 
support in this effort. Thus, the relationship between local 
research units and sponsor personnel is strengthening in an attempt 
to answer the questions posed in years 3 and 4. 
Conclusion ; 

During the initial years (1 and 2) of a program with many well 
defined components, the major focus will be on how well these 
con^nents have been implemented. Classical pre- post designs do 
not appear appropriate, at this point in time, and only become 
relevant in later years (3 through 6) when the effectiveness and 
intact of the program need to be assessed. 

Figure 1, indicates that the need for model specific data 
decreases over time, while the demand for qualitative and/or descrip- 
tive data reaches its peak between years 3 and 4. In contrast, the 
demand for quantitative data, in particular, standardized achieve- 
ment data will increase in years 3 through 6. 



Figure 1 
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